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Introduction 



Terminal reporting has become a matter for concern now that 
many projects are coming up to the three-year cutoff point. It Is In- 
teresting to note that the first USOE guidelines (1965) that were given 
In the manual for project applicants did not even make mention of 
terminal considerations, and the 1966 and the 1967 revised versions 
do mention an end-of-grant report but give only the barest suggestions 

as to what it should contain. 

This report is an effort to determine the' contents of 137 
randomly selected terminated planning and operational ESEA Title 

III grants. 

One should take into account the Met that most of the 137 pro- 
jects represented early-funaed projects. Later projects, beginning 
with the second year, have displayed a more sophisticated approach. 

Also, our paper analysis should be considered in light of the 
great difficulty of judging field success — where it really counts of 
the project as opposed to the paper picture. Members of the PACE 
national study team have been impressed by what they found in the 
field as compared with the project proposals. So one must view find- 
ings from analyses of terminal reports, no matter how carefully 
undertaken, with some skepticism. 
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With an understanding of the limitations of our "paper 



tiger, 
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nevertheless, we believe it represents a useful dimension of this 
second national effort. Every project is required to submit a termi 
nal report. From these efforts, one should be able to glean some- 
thing about how public money has been used to improve public educa 



tion. 

A four-man study steam was coordinated by Charles F. Mar- 
tin, assistant professor of education. University of Kentucky, who also 
did a major portion of the final analysis. The team members brought 
varied and well-rounded backgrounds to the 9 tudy, with two members 
having a total of 32 years of public school teaching and administration, 
and the other two having predominately college teaching and research 
experience. The other three team members were: Russell Bowen, 

graduate assistant. Department of Curriculum and Instruction, Univer- 
sity of Kentucky; Art Lucky, psychologist. State Department of Health, 
Frankfort, Kentucky; and John R. Payne, assistant professor of edu- 
cation, Virginia Polytechnic Institute. 

This study is sponsored by an ESEA Title III grant, to the Cen- 
ter for Effecting Educational Change, Fairfax County, Virginia, which 
subcontracted the assignment to the University of Kentucky's Research 



Foundation. 

Richard I. Miller 
Director of Study 
November 15, 1968 



ANALYSIS AND EVALUATION 



OF 

137 ESEA TITLE III PLANNING AND OPERATIONAL GRANTS 
The general objectives of the study were to determine the over- 
all influence and impact on education of a sample of terminated ESEA 
Title III Operational and Planning Grants. The five major areas in the 
evaluation instrument included: project characteristics, project ac- 

complishments, provisions for continuation, project design, and final 
appraisals. (A copy of the evaluation instrument is included as Appen- 
dix A. ) Specifically, the study sought to: 

1. Provide a statistical report and summary of a sample 
of terminated Title III Operational and Planning Grants. 

2. Determine whether individual project objectives were 
identified and achieved. 

3. Determine whether PACE participation resulted in edu- 
cational changes and improvements in school districts, 
including any evidences of direct influences upon stu- 
dents. 

4. Determine whether school districts were motivated to 
continue their programs through local funding and 
initiative. 

5. Determine as much as a paper analysis will allow 
whether. the expenditures, efforts, and intensive 
activity generated by PACE are justified in terms of 
outcomes. 
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Planning and orientation sessions were held to acquaint team 
members with objectives of the study. 

1. The instrument developed for analysis of the reports 
covered five major areas and several related categories. In addition, 
the terminated planning grants were analyzed through use of an adden- 
dum covering the three specific areas of n^eds assessment, program 
development, and possibilities of converting the planning grant into an 
operational grant. The instruments provided 111 individual checklist 
opportunities for responses, plus 49 items in the addendum that were 
applicable only to planning grants. 

2. The sample consisted of 94 terminated planning grants 
and 43 terminated operational grants. Thirty-five states were repre- 
sented in the planning grants and 28 in the operational grants. The 
states included all of the geographical areas of the nation. 

3. The analysis consisted, in the first instance, of identify- 
ing terminated planning and operational grants and separating them 
from the other types of grants. This task was unexpectedly time con- 
suming and somewhat difficult due to the fa,ilure of the many reports 
to properly and fully complete the official application and report 
forms. The individual planning and operational grants then were 
given a code number and the analysis began. 

Analysis was accomplished by application of the guidelines in- 
strument. Following the completion of the 94 planning and 43 
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operational grants, various statistical summaries were compiled. 
These will be presented in the following section of the report. During 
the actual analysis the team members conferred freely concerning 






problems of identification, meaning, intent, content, and so forth. 

This report will consist of the following six sections: statisti- 

cal summaries, project accomplishments, evaluation procedures, 
major problems, overall appraisal, and recommendations. 



STATISTICAL SUMMARIES 



This section includes a number of items that are included to 
assist the reader in understanding the S9,mple of 137 projects upon 
which this study is based. 

These items are: 

Number of projects in ecich submission period 
A financial summary 

School enrollment and staff involvement 
Type of project 
Focus of activity 

Identification of projects by states 

Scope of project 

Area served 

Types of activity 

Types of project 

Provisions for continuation 

Number of Projects in Each Submission Period 
For planning grants 

The study covers three funding periods — fiscal year 1966, 1967, 
and part of 1968. From the first period in 1966 (projects numbering 
1-733), 11 projects are analyzed and processed; from the second peri- 
od of fiscal 1966 (projects numbering 734-1723), 47 projects are 
analyzed; and the third period (projects numbering 1724-2726), in- 
cludes 26 projects. Thus a total of 84 of the 94 planning projects 
analyzed come from the 1966 fiscal year funding period. 

For fiscal year 1967, eight projects are analyzed. Threje are 
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from the first 1967 funding period (projects numbering 2727-3152), and 
five are from the second 1967 funding period (projects numbering 3153- 
4487). The sample includes two projects from the first period of fis- 
cal year 1968, encompassing projects numbering 4485-5795. 

The large number of planning grants in the 1966 fiscal year 
(91.6 percent as compared to only 9.4 percent for both 1967 and 1968) 
reflects the fact that ESEA was implemented in 1965 and became opera- 
tional in 1966. The impetus "to get in on a good thing, " the relative 
ease with which early planning grants were approved and funded, and 
the pressures from the public and educational establishment for school 
participation were factors that prompted submission of several hundred 
project proposals during the first funding period, or very shortly after 

ESEA became a law. 

For operational grants 

The first period of 1966 included 10 terminated operational pro- 
jects; the second period of fiscal 1966 was represented by 17 projects; 
and the third period of 1966 included 13 projects. Thus a total of 40 
projects of the 43 evaluated by the study team originated in fiscal 1966 
or immediately following enactment of the ESEA in 1965. The sample 
included one project from the first period of fiscal 1967, and two from 

the second submission period of 1967. 

The following table summarizes the data for both planning apd 
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operational grants with respect to the number of projects in each sub 
mission period. 



Fiscal 1966 


Planning 


Operational 


Total 


First period 


11 


10 


21 


Second period 


47 


17 


64 


Third period 


26 


13 


39 


Fiscal 1967 


First period 


3 


1 


4 


Second period 


5 


2 


7 


Fiscal 1968 


First period 


_2 


_0 


2 


Totals 


94 


43 


137 



One might ask about the validity of this sample in view of the 
significant improvement in the quality of proposals submitted during 
the second year. The question is a legitimate one, and the study team 
was aware of it in making their analyses and recommendations. 

A Financial Summary 

For planning grants 

The 94 Title III planning grant projects received a total of 
$5, 295, 006 during the funding periods encompassed in this report. Of 
this amount, $588, 982 was for non-federal support, largely state or 
local funds; and the federal support other than ESEA Title III totaled 
$53, 192. ‘Actual federal support for Title III for the three year period 
included in this study totaled $4, 649, 747. 
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For operational grants 

The total fiscal allocation for the 43 terminated operational 
grants was $4, 330, 579. Of this amount, $661, 335 came from non- 
federal sources. The actual Title III allocation totaled $3,401,807, 
while federal support from sources other than Title III totaled 
$265, 947. Due to arithmetical errors in the financial section of 
some individual projects, the total in those reports do not balance 
exactly. 

The following table summarizes the data for both planning and 



operational grants with respect 


to project 


support. 




Type of 


ESEA 


Other 


Non- 


Total 


proiect 


Title III 


federal 


federal 


94 planning 


4‘, 649, 747 


53, 192 


588, 982 


5, 295, 006 


43 operational 


3.401,807 


265, 947 


661, 335 


4. 330. 579 


Totals 


8, 051, 554 


319, 139 


1, 250, 317 


9, 625, 585 



School Enrollment and Staff Involvement 
For planning grants 

The 94 planning projects reviewed by the study team encom- 
passed a geographic area containing an estimated total school popula- 
tion of 3, 845, 018 students. (See table on page 11.) At the pre-kinder- 
garten level, 14, 383 public and 1, 197 non- public school students were 
located in the geographic area served by this sample. The total num- 
ber of kindergarten pupils served in the geographic area included 

1 










tummm 









n tig Mu i . wyi 



- 10 - 



156, 755 public school pupils and 11, 875 non-public school pupils. In 
grades one through six, 1, 532, 506 public school pupils ^nd 275, 611 
non-public school pupils; and fpr grades seven through twelve, 

655, 724 public and 88, 517 non-public school pupils were totaled. 

The distribution of participants, according to race, is; 

2, 154, 760 white; 123, 947 negro; 1 1 , 723 American Indian; apd 61, 997 
other non- white — for a total of 2,352,431. (See table on page 13.) 

For operational grants 

The 43 operational projects encompassed a geographic area 
containing an estimated school enrollment of 1, 494, 192 students. (See 
table on page 11.) At the pre-kindergarten level, an estimated 1,404 
public and 390 non- public school students were in the geographic area 
included in this sample. The estimated total number of kindergarten 
pupils in the geographic area included 26, 303 public school pupils and 
1,663 non-public school pupils. In grades one through six, 745,027 
public school pupils and 109, 884 non-public school pupils; and for 
grades seven through twelve the total was 524, 695 public school and 
42, 050 non-public school pupils. 

Again, the estimated number of pupils actually served by the 
projects is much smaller than the total geographic population. (See 
page 11.) At the pre-kindergarten level, 1,260 public and 184 non- 
public school pupils were served. At the kindergarten level, 11, 195 
public and 274 non-public school pupils were listed; for grades one 
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through six, 226,361 public and 14,817 non-public school students were 
listed; and for grades seven through twelve the total was 156, 731 

public and 4, 192 non-public school pupils. 

The distribution of participants, according to race, is: 338, 123 

white; 47, 420 negro; 3, 367 American Indian; 1, 789 other non-white 

for a total of 410, 483. (See page 13. ) 

The preceding tables summarize data on school enrollment and 

staff involvement; 

The reader should not confuse these "ball park" figures with 
what might actually represent the scope of the project. 

•phe figures do represent some very rough indication of the 
maximum number of pupils and students that might be influenced by 
the project under the most favorable conditions. But this c^uantitative 
dimension must not be confused with quality, £e£._sje. 



Type of Project 

Of the 94 planning grants, 53 projects were designated as in- 
novative, 32 as exemplary, and two as adaptive. (These categories 
follow those set up by the USOE guidelines.) A total of seven proposals 
submitted failed to designate or identify the type of project. 

Of the 43 operational grants, 26 were designated as innovative, 

14 as exemplary, and three as adaptive. 
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The following table summarizes the data on this item: 

Type 94 Planning 43 Operational Total 



Innovative 
Exemplary- 
Adaptive 
No description 
Totals 



53 

32 

2 

__7 

94 



26 

14 

3 

43 



79 

46 

5 

7 

137 



These categories remain elusive. The first national study de- 
fined an educational innovation as "a new or different concept, methodol- 
ogy, organization, or program that is systematically introduced into 
the classroom, school system and/or the State as a whole." 

In this sense many projects appeared to be an innovation, but 
many others seemed to be adaptive. F ew could be judged as exem- 



plary. 



Focus of Activity 

For planning projects 

A large majority of the projects — 83 in number— —concentrated 
on program planning, and 18 projects conducted pilot studies as as- 
sociated activities. (Again, these categories were taken from USOE 
guidelines.) Eight projects planned construction, while seven were 
designated as operational, and one for remodeling. One project made 
no response in this category. (The number of activities listed exceeds 
the total number of projects because some projects sponsored more 
than one major activity. ) 
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For operational projects 

Twenty-five operational projects were judged to have planning 
as a major activity or program focus. This finding is a little unusual 
in that one would expect most program planning to take place during 
the planning grant phase of the program. Thirty- seven projects con- 
centrated on program operation, while 10 conducted various types of 
pilot activities. Four were devoted primarily to construction of facili- 
ties, three to remodeling, and two to planning of construction, (The 



number of activities exceeds the total number of projects because 



several projects focused on more than one activity.) 



The following table summarizes the findings; 



Activity 



Planning Operational 



Planning of program 83 25 

Planning of construction 8 2 

Conducting pilot programs 18 10 

Operation of program 7 37 

Constructing 0 4 

Remodeling 1 3 

Not identifiable 1 

Totals 118 81 



Total 



U08 

10 

28 

44 

4 

4 

1 

199 



Identification of Projects by States 
For planning grants 

The 94 projects were located in 35 of the 50 states. The eight 



northeast states provided 27 of the projects used in the study; the mid- 
west consisting of eight states provided 21 proposals; the southern 
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region consisting of nine states provided 19 projects; an^ the western 
region consisting of nine states provided 19 projects. 

California led in number of projects with nine; Pennsylvania 
followed with eight, and Michigan, Illinois and New Jersey each had 
five projects. Three states had four projects in the sample, seven 
states had three projects each, eight states had two projects each, and 
13 states had only one project. 

For operational grants 

The 43 projects were located in 28 of the 50 states. The west- 
ern area accounted for 12 projects; the south had 12 projects; the 
east had 10 projects; and the noorth had nine. 

California led in number of projects with six. Eight states had 
two projects, and 19 others had a single project. 



The following table- summarizes the distribution by states: 
Distribution of Planning and Operational 



Grants by State s 



States 



Planning Operational 



Total 



Alabama 

Alaska 

Arizona 

Arkansas 

California 

Colorado 

Connecticut 

Delaware 

District of Columbia 

Florida 

Georgia 

Hawaii 



1 

1 

0 

1 

9 

1 

2 

2 

0 

3 

4 
0 



0 

0 

0 

1 

6 

0 

1 

0 

0 

2 

0 

0 



1 

1 

0 

2 

15 

1 

3 
2 
0 
5 

4 

0 









mmm 



States 



Idaho 
Illinois 
Indiana 
Iowa 
Kansas 
Kentucky 
^ i. ' Louisiana 
Maine 
Maryland 
Massachusetts 
Michigan 
Minnesota 
Mississippi 
Missouri 
Montana 
Nebraska 
Nevada 

New Hampshire 
New Jersey 
New Mexico 
New York 
North Carolina 
North Dakota 
Ohio 

Oklahoma 

Oregon 

Pennsylvania 

Rhode Island 

South Carolina 

South Dakota 

Tennessee 

Texas 

Utah 

V ermont 

Virginia 

Washington 

West V irginia 

Wisconsin 

Wyoming 

Unidentified 



Planning Operational 



1 

5 

1 

0 

2 

1 

3 

0 

1 

1 

5 

3 

0 

3 

1 

0 

0 

3 

5 

2 

3 
2 
0 

4 
0 
0 
8 
4 
0 
0 
2 
3 
1 
0 
2 
1 
0 
1 
2 
1 



1 

0 

0 

1 

2 

2 

0 

0 

1 

2 

1 

1 

0 

0 

0 

0 

0 

1 

1 

1 

2 

1 

1 

2 

1 

1 

2 

0 

1 

0 

0 

2 

1 

1 

1 

0 

0 

1 

0 

2 
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Total 



2 

5 
1 
1 
4 
3 
3 
0 
1 

3 

6 

4 
0 

3 
1 
0 
0 

4 
6 
3 

5 

3 
1 

6 
1 
1 

10 

4 
1 
0 
2 

5 
2 
1 
3 
1 
0 
2 
2 
3 




1 
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Scope of Project 

For planning grants 

The scope of the projects indicated that § majority (40) covered 
one district. Another 40 proposals concentrated on multi-district 
areas, with 26 in multi-districts of seven or more, and 14 were in 
multi- districts of six or less. Only one regional (projects covering 
more than one state) and one state project were included in the sample. 
Five of the projects sampled involved a few schools in one district 
while four concentrated only on one school in a district. Three pro- 
jects failed to indicate the scope or limitations of their projects by not 
responding to this item. 

For operational grants 

None of the terminated operational projects were national in 
scope nor did any of them cover an entire region or state. Eleven of 
the projects focused upon one district, 14 functioned in multi-districts 
containing two to six different school systems, and six grants operated 
in multi-districts of seven or more. Seven projects were located in a 
few schools within a single district, and two projects operated each in 
one school. The scope of the terminated operational projects compares 
favorably with the distribution of the Planning Grants in that the majo- 
rity were located in either one district or in multi-districts. Three 
projects failed to designate the scope of their projects, and were not 



included in the report. 
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The distribution of planning and operational grants is given in 



the following table: 
Scope 



Planning Operational 



Total 



National ^ 

Regional ^ 

One state 1 

One district 40 

Multi-district (2 - 6) 14 

Multi-district (7 +) 26 

A few schools in one district 5 

One school 4 

Not identifiable 3 

Totals 94 



0 

0 

0 

11 

14 

6 

7 

2 

43 



0 

1 

1 

51 

28 

32 

12 

6 

6 

137 



Area Served 



For planning projects 

Approximately 72 projects out of the 94 focused on urban areas 
while only 21 served rural areas. This is not surprising since the 
needs of inner-city and large urban area schools are a major thrust 
for PACE. Twenty-three of the projects served central city areas and 
12 served urban fringe areas, and 20 projects were operative in cities 
of 10, 000 or more, while 15 served cities of 2, 500 to 10, 000. 

Twelve projects focused on rural areas — communities of 1, 000 
to 2, 500 and only nine served areas designated as "other rural." Three 
projects failed to designate the area served by failing to respond to this 




particular category. 



- 21 - 



For operational projects 

A majority of the projects (18) served either the inner-city or 
fringe areas; however, an almost equal number were located in subur- 
ban areas with 12 projects serving cities of 10, 000 or more and five 
serving cities of from 2, 500 to 10, 000 population. Rural sections had 
a much smaller percentage of projects with six located in rural areas 
of from 1, 000 - 2, 500 population, and only two operating in rural areas 
smaller than 2, 500 population. As in the case of the planning projects, 
a large majority of operational projects were concentrated in the in- 
ner-city and urban areas. 

The following table summarizes the areas served by the 137 



projects: 



Area 

Urban 


Planning 


Operational 

•%' . 


Total 


Central 


23 


10 


33 


Urban fringe 


12 


8 


20 


Other urban (suburban) 


Cities of 10, 000 and over 


20 


12 


32 


Cities of 2, 500 to 10, 000 


15 


5 


20 


Rural 


1, 000 to 2, 500 


12 


6 


20 


Other rural 


9 


9 


11 


Unidentified 


__3 




3 


Totals 


94 


43 


139 
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Tvpes of Activity 

F or planning grants 

The types of activity engaged in by the various projects were 
varied. Not surprisingly, activities related to planning, surveying 
needs, designing new programs and visiting innovations were most 
frequent, with 36 projects giving this category as their prime activity. 
The second most prevalent activity consisted of services to local 
schools with 21 projects, and the third most frequently listed cate- 
gory — with 18 projects — was instruction or services to pupils in the 
arts, social studies, guidance and counseling, and various other 
activities related to student instruction. Ten projects concentrated 
on services to teachers and principals, mostly in the areas of in-ser- 
vice training, audio- visuals and instructional materials, and seven 
projects emphasized the installation of one or two innovations in a 
single school or two, while two projects failed to indicate their pri- 
mary focus. 

For operational grants 

A large majority of the 43 projects, however, concentrated upon 
instructional services to pupils in the arts, sciences, social studies, 
etc. , by providing mobile labs, ETV, visits to museums and guidance 
and counseling services. Six of the projects provided administrative 
planning, dissemination and developmental services to local school 







districts. 
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The emphasis on various phases of instructional services is to 
be expected. Seven of the projects provided in-service programs to 
administrators and teachers in audio-visual training, provisions for 
instructional materials, and demonstrations of programs. Only three 
programs out of a total of 43 projects surveyed involved the installa- 
tion of innovations in one or two schools, and no programs were con 
cerned with activities such as surveying needs, visiting other innova- 



tions, or planning for these activities. 

The following table summarizes the types of activities found in 
the 137 projects. (The total number of operational projects exceeds 
the 137 figure because a few projects were judged as having two or 



more major functions.) 

Type of Activity 



Planning 



a. Mostly instruction or ser- 
vices to pupils, such as 
arts, sciences, social stu- 
dies, mobile demonstra- 
tion, museum visits, ETV , 
outdoor camping, and 
guidance and counseling. 

b. Mostly planning, such as 
surveying needs, design- 
ing new programs, and 
visiting innovations 

c. Mostly services to several 
local school districts, such 
as administration, dissemi- 
nation, planning, and de- 
veloping 



18 



36 



Operational Total 



36 



54 



0 



36 



6 27 



21 






Type of Activity 
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Planning Operational T otal 




d. Mostly services to teachers 

and principals— services 
such as instructional mate- 
rials, audio-visual, demon- 
strations, and in-service 
training 10 

e. Mostly installation of one or 
two innovations in one or two 
schools— - innovations such as 
ITA, computer-assisted instruc- 
tion, teaching aides, parept in- 
volvement, job placement, and 

new courses 7 

f. Not indicated 

Totals 94 



7 17 



3 10 

52 146 



Types of Project 

For planning projects 

The 94 PACE planning projects were almost evenly divided in 
terms of focus: Forty- six concentrated upon a single idea or program, 

while 45 were planning to serve as supplementary centers usually with 
several programs or activities. Three projects failed to indicate an 
area of concentration. 

For operational pfoiects 

Twenty- seven of these projects focused upon a single idea or 
program, or about 62% of the total number. The remaining number 
of grants (16) concentrated on organizing or providing various types 
of supplementary educational centers. 




!np 
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The following table summarizes the types of projects found in 
the 137 projects: 



Focus 


Planning 


Operational 


Total 


Upon single idea or program 


46 


27 


73 


Supplementary center program 


45 


16 


61 


No indication 


__3 


_0 


3 


Totals 


94 


43 


137 



Addendum for Planning Grants 

As was noted on page four of this report, a special addendum 
covering the specific areas of (1) needs assessment, (2) program 
development and (3) possibilities for converting planning grants into 
continuation grants, was used in analysis of the 94 planning grants 

sampled. 

The study team determined where a needs assessessment study 
had been made as an aspect of the planning grant. Thirty- six projects 
were judged to have done so, and 57 projects had not made a needs 
assessment study. 

A second part of this criterion determined what community and 
school factors were used in those 36 projects that did conduct an as- 
sessment study to determine their needs. The two leading community 
factors were (1) rural-urban composition and (2) socio-economic 
class, with 12 responses each. Eleven projects indicated that 
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employment patterns were important community considerations, while 
nine projects listed attitudinal considerations as the most prominent 
community factor used. 

Among the other community variables used by the study team 
were median family income, median grade attainment, and horizontal 
mobility, with four each; and the school variable, per ipupil expendi- 
ture with five responses. Some six projects indicated that pupil and 
community needs were important community factors, however, this 
appears to be a dual category including both community and school 

factors. 

Turning to the school factors used in the assessment studies, 

14 projects used composition of the student body, student achievement 
measures and the socio-economic class structure of the school system 
were used by eight projects each. Five projects used study needs as a 

school factor in needs assessment. 

Community and school variables used by the 36 (out of 94) pro- 
jects that conducted needs assessment studies are summarized in the 
following table: 



o' P 






wmmm 



- 27 - 



Community factors' 






Rural-urban composition 
Socio-economic composition 
Employment patterns 
Attitudinal considerations 
Per pupil expenditure 
Median family income 
Median grade attained 
h. Horizontal mobility 
Other 



c . 

d. 

e. 

f. 

g- 



1 . 



School factors 

a. Composition of student body 

b. Socio-economic class structure 

of school system 

c. Student achievement measures 

d. Other 



Frequency used 

12 

12 

11 

9 

5 

4 

4 

4 

3 

F requency used 

14 

8 

8 

3 



The number of community factors used exceeds the total 
number of projects because several projects used more than one 
procedure to assess needs. 




o 
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PROJECT ACCOMPLISHMENTS 



The second category appraised by the study team covered 11 

items related to project accomplishments. 

Accomplishments on the 137 projects were rated in terms of 

what the project authors said was achieved. 

The 11 items were: 

Research 

— — - Product development 

Skill development 
Process development 
Program development 
Pupil relations 
Community relations 
In-service programs for teachers 
Demonstration 
Dissemination 
Implementation 
Provisions for continuation 



Research 

Twenty-three terminal planning project reports, or 24 percent, 
had some type of research design that was related logically to the pro- 
gram. Fifteen of these 23 projects had accomplishments that appeared 

significant in terms of potential contributions. 

In three cases the accomplishments were considered to be over- 
stated, and in 11 projects the accomplishments werfe defined as 



’’ordinary." The accomplishments of two out of the 23 projects were 

rated as "trite or insignificant. " 

For 76 percent of the terminal reports, however, no research 

design or research attention was evident. 

Nineteen terminated operational project reports, or 44 percent, 
were judged as having research-oriented programs, or as having used 
research methodology in planning, operational procedures and/or 
evaluation. Research procedures in 18 of the 19 projects appeared to 
flow orderly and logically from the program, but in only one project 
did the claimed accomplishments in terms of research appear to offer 
significant contributions. 



Product Development 

This area refers to the development of new programs and ma- 
terials such as curriculum innovations or aids for the handicapped. 

Activities in 45 planning projects fitted into this category, and 
22 of this number were judged to have accomplishments flowing log- 
ically from the programs. Fifteen of the 45 were credited with having 
accomplishments that were significant in terms of potential contribu- 
tions. Accornplishments in three of the 45 programs were judged as 
being over- stated, and the accomplishments in one program were 
judged to be trite or insignificant. 

Product development activities in 16 operational projects 
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seemed to originate and to flow logically from the project's overall 
program, and three projects in product development presented ac- 
complishments that seemed capable of making significant contribu- 
tions. One claim made by the project authors was judged to be ":trite 
or insignificant"" by the reviewers. 

Skill Development 

Activities in eight terminal planning reports could be classified 
as skill development, which included activities such as new mathema- 
tics, foreign languages, and the Initial Teaching Alphabet (ITA). Five 
of these eight were rated as flowing logically from the program, and 
three of the activities presented accomplishments judged to be signi- 
ficant. 

Fourteen operational projects had skill development in subject 
matter areas such as reading, mathematics, and the social sciences 
as their major goal. Of this number, 13 projects were credited with 
flowing logically from the program. One other project was judged to 
have made claims that were "trite or insignificant" and not likely to 
make a significant contribution. 

Process Development 

Eleven projects stressed process development such as inquiry 
training and critical thinking. Five of the 11 projects were judged to 
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have accomplishments that could make potential contributions in pro- 
cess development; five were appraised as claiming accomplishments 
that were "ordinary;” and only one project was judged to have made 
an orderly or "flow" transition from the project. Four out of the 43 
operational projects had some form of process development as an inte- 
gral part of their total efforts. These four programs were judged to 
flow logically from the project itself; however, none was judged as 
likely to make significant contributions to education in their area. 

Program Development 

Some 52 planning projects featured various types of program 
developments such as computer assisted instruction, non-graded 
schools, special science programs, and team teaching. One program 
cLCcomplishment was judged to be "overstated;" ten were appraised as 
being "ordinary;" 18 programs seemed to flow logically from the pro- 
ject; and 22 were judged significant in terms of potential contributions. 

Four projects included some aspect of computer technology as 
a major dimension. One focused on computer assisted instruction as 
a supplement to the regular teaching methodology, and other two pro- 
jects utilized computer assistance in pupil accounting, guidance ser- 
vices and personnel recording. The small number of projects involving 
computer technology confirms the fallacious nature of some early esti- 
mates that said the bulk of PACE resources was going into hardware. 
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Seven projects used various media as integral parts of the pro- 
ject design and operation. Two were instructional television (ITV) pro- 
jects: one used rural and local artifacts in a museum setting as a part 

of ITV teaching, and the other used ITV to present various cultural en- 
richment programs. Video tapes and other materials were used in 
one project, and two projects constructed models to demonstrate new 
programs to students and teachers. Another project planned to develop 
a system for classification, location and retrieval of specific instruc- 
tional materials. 

For the operational projects, program development in such 
areas as team teaching and organization of non-graded schools, etc. , 
was evident in 23 out of the 43 projects. Twenty-one of the 23 were 
appraised as being consistent with the proposal, and two were rated 
as offering potentially significant contributions to their field. 

Pupil Relations 

Twenty programs stressed pupil relations, referring primarily 
to programs that directly involved pupils. Ten of these programs ap- 
peared to flow logically from the project, and five projects were judged 
significant in terms of potential contributions. Five of the 20 were ap- 
praised as "ordinary. " 

Operational programs featuring pupil relations were scarce, as 
was the case with this category for planning reports. For the 
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operational grants, eight out of 43 projects were directly designed to 
further the relations between the school and the pupils outside the regu- 
lar curriculum. Two of the eight were judged to flow directly and log- 
ically from the program, but six were appraised as making claims 
that were "trite or insignificant. " 

Community Relations 

A relatively large number of planning projects — 67 — claimed 
accomplishments in community relations. Three claims for accom- 
plishments were judged as "overstated," 13 were classified as "ordi- 
nary," and one was considered "insignificant." Twenty-eight projects 
listed accomplishments in community relations that appeared to flow 
logically from the stated program, and 22 were appraised as making 
significant contributions in terms of community relations. Many of 
the claimed accomplishments, however, are judged as superficial and 
were primarily "paper" involvements rather than meaningful commu- 
nity participation. Only four projects were judged as including in a 
significant manner the community, local teachers, and the pupils. 

Thirty-five operational projects claimed they had achieved ac- 
ceptable community relations in the areas where their projects func- 
tioned; however, the review team concluded that the claims made by 
the projects were only "trite or insignificant" in 23 out of the 35 cases, 
and did not possess a great deal of innovative ingenuity. Nine others 
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W6r6 6 V 3 .lu 3 .ted 3 S being only **ordin3ry," while only three were ere 
dited with h 3 ving promising contributions tow3rd better community 
rel3tions. 



In - service Progr3ms for Te 3 cher s 
Sixty- six termin 3 l pl 3 nning project reports included in-service 
tr 3 ining 3ctivities for te3chers, 3dministr3tor s, 3nd/or st3ff mem- 
bers. Accomplishments in 16 of the 66 projects were 3ppr3ised 3S 
being "ordin 3 ry;" three were "over st3ted;" 25 progr 3 ms seemed to 
flow logic3lly from the projects; 3nd 22 achieved signific3nt results 
in terms of possible tr3nsfer. 

Four projects concentr3ted on 3 V3riety of in-service tr 3 ining 
progr 3 ms for te 3 chers 3 nd other profession3l school personnel. These 
projects included retr3ining of te 3 chers by outside "experts" 3S well 
as instruction by co-workers who had t3ken speci3l courses. In sever- 
al inst 3 nces, teachers were included in the planning st 3 ges through 
conferences and discussion groups. In a few cases speci3l emphasis 
was given to training teachers prior to major curriculum changes. 

The frequent inclusion of in-service educ 3 tion in planning pro- 
jects as well as the appearance of four projects that were concentrating 
on this area is encouraging. Too few projects, however, extensively 
involved local teachers in strategies related to planning, dissemina- 



tion, and implementation of an innov3tion. 
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The appraisal team found that 26 operational projects apparently 
provided for some type of in-service training. Of this number, 11 pro- 
grams were rated as flowing naturally from the program structure, 
and six were judged capable of making significant contributions to in- 
service education. The claims of nine projects, however, were dis- 
credited and their accomplishments were judged "trite or ordinary." 

As judged from the reports, not a great deal in the way of in-service 
education was accomplished. 



Demonstration 

Twenty-three planning projects featured provisions for demon- 
strating programs, such as team teaching and others. Of this number 
eight claims for accomplishment were appraised as "ordinary," seven 
flowed logically from the program, and eight were appraised as offer- 
ing significant contributions. 

Nine operational projects described programs demonstrating 
innovations, such as team teaching. Of this number, the reviewers 
concluded that four flowed logically from the program or were an inte- 
gral part of the projects, four appeared capable of making significant 
contributions, and one was judged to be "trite or insignificant. " 

Piss emination 



Inall, 153 accomplishments in dissemination were claimed, which 
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indicates that some projects used more than one method of dissemina- 
tion. 

Accomplishments of three projects out of 94 were judged as 
"overstated," 55 of the claims were classified as "ordinary," and 
eight were judged to be "trite or insignficant. " Fifty-four were judged 
to flow logically from the project program, and 27 were rated signifi- 
cant in terms of potential contributions. One program’s claim of ac- 
complishment was judged to come from a source that did not seem re- 
lated to the program. 

All 43 terminated operational projects said that efforts were 
made to disseminate the information about their project. While a 
rather wide assortment of dissemination strategies were employed, in 
only 12 situations did the procedures seem to flow logically from the 
program, and only nine projects presented accomplishments that ap- 
peared likely to produce significant contributions. Nineteen projects 
laid claims to dissemination that were judged to be "ordinary, " and 
claimed accomplishments of three projects were judged as "trite or 
insignificant. " The review team discredited the effectiveness of al- 
most 50 percent of the claimed dissemination efforts, and they believed 
that less than 20 percent of the claimed accomplishments were signifi- 



cant or potentially promising. 
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Implementation 

Sixty projects out of the 94 planning grants were rated on the 
extent of implementation achieved by the project. The appraisal team 
judged claims for success in implementation were "overstated" by one 
project, "ordinary" for ten others, and "trite or insignificant" m four 
projects. In the case of 24 other projects, implementation seemed to 
flow logically from the program, and in 21 projects implementation of 

accomplishments was rated as significant. 

Eighteen of the operational projects indicated that plans for im- 
plementation had been built into their projects. After evaluation, the 
review team judged that six out of the 18 projects had plans for imple- 
mentation that likely were successful. Three of the projectk' imple- 
mentation plans were judged to flow logically from the program itself; 
however, eight other plans were judged as being "ordinary, " while one 
was considered "trite or insignificant. " Fifty percent of the plans for 
implementation given by the projects were judged as capable of suc- 
cess, and another 50 percent were of such poor quality that failure 

seemed assured. 



Provisions for Continuation 



For planning projects 

Fifty- nine out of the 94 grantees indicated they have given some 



consideration to grant continuation and support by means other than 
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ESEA Title III funds, and 35 projects indicated that no consideration 
had been given for grant continuation. 

Of those wanting to continue, 30 projects appeared likely to 

<r 

succeed in this respect. Fifteen other projects planning or wanting to 
continue were judged as unlikely to succeed. Five projects submitted 
quite sketchy plans for continuation, and in six instances, the plans 
were too nebulous to be appraised by the analysts. Three projects 
failed to respond in this category. 

For operational projects 

A rather large number — 33 out of the 43 — indicated that speci- 
fic plans had been, or were being made, for project continuation after 
termination of funds. Seven projects indicated that they had formulated 
no plans for continuation, and three projects made no response. 

In rating continuation likelihood, 12 of the 33 projects that had 
plans seemed likely to make it. Eleven presented continuation plans 
that were judged as questionable, five had plans that were at best 
"quite sketchy," and ten projects had plans that were too nebulous to 
appraise or evaluate. From this analysis, probably 12 to 15 projects, 
at best, will continue, through local efforts and funding, either in part 
or in toto. 



The following table summarizes the provisions for continuation: 
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Question 
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Planning Operational Total 



Does the project give at 
least some consideration 
to continuation by means 
other than support by ESEA 



Title III? 


Yes 


59 




No 


35 




No response 




Totals 




94 



33 

7 

_2 

43 



92 

42 

3 

137 



If yes, are these considera- 
tions (plans) 

a. well planned and likely to 



succeed 30 

b. questionable 15 

c. quite sketchy 5 

d. too nebulous to appraise 6 

e. no response 

Totals 59 



12 


42 


6 


21 


5 


10 


10 


10 


_g 


3 


33 


92 



evaluation procedures 



The first guidelines did not give evaluation the attention that 
was the case in later editions, yet some mention was made of it and 
some expectations for evaluation were evident. 

This section covers these items: 

Types of evaluative proc idures used 
Relationship between evaluation and overall project 
Cost of evaluation 

Adequacy* of evaluation procedures 

Types of Evaluative Procedures Used 
For planning grants 

Thirty-two projects stated that surveys (questionnaires) were 
the key evaluative methods used. Only four projects indicated that re- 
search design procedures figured in the evaluative techniques, and 
four other projects used pre- and post- tests. Fifteen grantees listed 
interviews as the main evaluative procedure, one listed opinions, four 
listed outside consultants, one conferences, two listed time studies, 
and one statistical analysis. But 45 projects gave no evidence or failed 
to respond to this question. 

For operational grants 

The survey or questionnaire technique reportedly was used by 
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17 projects, and 16 projects used pre- and post-test evaluative proce- 
dures. Nine used interviews, two opinion surveys or polls, and four 



used as the methodology and instrument for evaluation the official 



terminal reporting form required of ending Title III Grants. 

The following table summarizes the findings on types of evalua- 



tive procedures used: 





Type of procedure” 


Planning 


Operational 


a. 


survey (questionnaire) 


32 


17 


b. 


interview 


15 


9 


c. 


research design 


4 


4 


d. 


pre-test, post-test 


4 


16 


e. 


outside evaluators 


4 


0 


f. 


time studies 


2 


0 


g- 


opinion 


1 


2 


h. 


conferences 


1 . 


0 


i. 


statistical analysis 


1 


0 


j- 


no response or not indicated 45 


__0 


Totals 


109 


48 



Total 



49 

24 

8 

20 

4 

2 

3 

1 

1 

45 

157 



Relationship Between Eva luation and Overall Project 
For planning grants 

Eighteen projects were judged as having evaluation integrally 
related to the overall project; and 20 had evaluation as only a part, or 
slightly related, to the overall project. Thirty- seven projects had no 
particular relationship between the evaluation and the overall project. 



"“^Some projects used more than one evaluative procedure 
thereby accounting for the numbers in the above table exceeding the 
total number of projects in the sample. 



o 
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and in one case evaluation was forced to meet the overall characteris- 
tics and nature of the project. Eighteen projects failed to respond or 
indicate any relationship between evaluation and the overall project. 

For operational g[rants 

Twelve of the 43 operational projects included evaluation as an 
integral part of their projects, and 14 projects had some provisions 
for evaluation as a part of their overall project planning. Another 14, 
however, were appraised as having no evident relationship between 
evaluative procedures used and the overall project. In one project, 
the evaluation had been forced to correspond to the project's character- 
istics, and two projects made no response in this area or had such 
poor provisions for evaluation that they were incapable of being an- 
alyzed. 

The following table summarizes the relationships between 
evaluation and the overall proj 
Type of relationship 

a. Not any particular rela- 
tionship or very casual 

b. A part 

c. An integral part 

d. Evaluation appears forced 
to need the guidelines re- 
quirement 

e. No response 

Totals 



Planning 


Operational 


Total 


37 


14 


51 


' 20 


14 


34 


18 


12 


30 



1 


1 


2 


18 


__2 


20 


94 


43 


137 






! 
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Cost of Evaluation 

So few of the 137 planning and operation^ projects listed ex- 
penditures for evaluation that any statistical summary is meaningless. 
This omission emphasizes once again the lack of concern and under- 
standing of the importance of evaluation. The failure to formulate 
plans for evaluation is a major characteristic of a large majority of 
the reports, and this matter is considered in the recommendations. 

Adequacy of Evaluation Proce dures 
For planning projects 

Only 30 of the 94 projects were judged as having adequate 
evaluation procedures in the project design. This may be contrasted 
to 31 projects which gave no evidence whatso ey^ of evaluation proce- 
dures. Eleven projects indicated some provisions for evaluation, and 
eight gave a little evidence of evaluative considerations. Fourteen pro 
jects failed to respond to this category. Of the 94 projects, 49 gave 
indications of planning for evaluation ranging from little to much; 45 
gave no indication of evaluation, or they failed to respond in this area. 

For operational projects 

Only five of the 43 operational projects presented adequate 
evaluative procedures in their project designs. Another 20 projects 
presented some evidence that efforts or provisions for evaluation had 
been considered. Ten projects were judged as giving scant attention 
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or paying "lip service" to evaluation, and ^ix others gave jio evidence 
of provisions for evaluation. Two projects made no response in this 



area. 



A little over eight percent of the projects had made plans and 
provisions that promised to be adequate for evaluation of their pro- 
jects; about 70 percent (or 30) had done a little, and about 13 percent 

had not bothered with evaluation at all. 

The following table summarizes the adequacy of evaluative prO' 

cedures: 

Degree of adequacy Planning Operational 



a. much 

b. some 

c. little 

d. no evidence 

e. no response or not indicated 

Totals 



Total 



30 


5 


35 


11 


20 


31 


8 


10 


18 


31 


6 


37 


il 


_2 


16 


94 


43 


137 



i 



MAJOR PROBLEMS 



The study team judged each project in terms of this question: 
"From your study of the report, what were two or three major prob- 
lems that seemed to be implicit or explicit? 

F or planning projects 

Some 20 individual problems were identified, and two major 
problems were: (1) financing the project, mentioned 12 times, with 

the primary concern being that of finding additional funds for project 
continuation; and (2) problems associated with public relations, 
listed 10 times, which included matters such as bringing about public 
awareness of the projects and their intended accomplishments. 

Five other problems were identified, although mentioned less 
frequently but were still considered important. These were: (1) fail- 

ure to formulate clear objectives, listed eight times; (2) lack of plan- 
ning, listed seven times; (3) lack of qualified personnel, also listed 
seven times; (4) problems related to implementation, mentioned six 
times; and (5) lack of official cooperation from the schools, also 
mentioned six times. 

Of the 13 other problems that were gleaned from the 94 reports, 
only one occurred as many as three times, while four were mentioned 
two times, and eight were listed only one time each. 
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Twenty-nine projects, or about 30 percent, were judged as 
having no major or minor problems, and 11 projects were so vague 
or contained so little information that no problems could be identified. 
For operational projects 

Again, the problem of grant continuation led all others in fre- 
quency, with 13 projects giving it a major status. In all, 15 problems 
were identified. Next wa*s the problem of maintaining and increasing 
student interest, which was mentioned five times. The following prob- 
lems were listed four times each: (1) public relations or selling the 

pubb'-- and schools on the project, (2) the problem of overextension 
or attempting to do too much with their limited resources, and (3) the 
problem of overspending. 

Several projects felt that they were in danger of exhausting 
their budgets. Lack of adequate planning was listed as a major prob- 
lem in three instances. Four other problems were listed two times 
each, and five additional problems each were listed once. 

Only four projects were judged as having no major problems, 
and four other projects did not contain sufficient information to ascer- 
tain if they did or did not have problems. 

The following table summarizes the major problems found in 



the 137 projects: 



Problem 



Planning 



Operational 



Total 



a. Grant continuation 12 

b. Public relations 10 

c. Failure to formulate 

clear objectives 8 

d. Lack of planning 7 

e. Lack of qualified personnel 7 

f. Implementation 6 

g. Evaluation 3 

h. Poor facilities 2 

i. . Student interest 2 

j. Overextension of effort 1 

k. Dissemination 1 

l. District cooperation 1 

m. Inadequate leadership 1 

n. Overspending 0 

Totals 61 



13 

4 

0 

3 
2 
2 
0 
0 

5 

4 
0 
0 
2 

__4 

39 



25 

14 

8 

10 

9 

8 

3 

2 

7 

5 

1 

1 

3 

4 

100 



OVERALL APPRAISAL 



This overall appraisal by the study team was based upon 
several approaches. The appraisal included a composite summary 
of the checklist instrument as well as a complete introspective analy- 
sis of each project. Included as criteria to be rated were such items 
as completeness planning, identification of objectives and methodology, 
involvement of local persons as well as outside consultants, provisions 
for continuation, and probable overall success of the project. 

For planning projects 

The study team made an overall appraisal of the project, as 
gleaned from the report. Fourteen were rated as outstanding, 37 as 
good, 19 average, 18 poor, and 6 very poor. Thus, 51 (or 54. 2%) 
of the 94 were rated either outstanding or good. Nineteen (or 21%) of 
the projects ranked average, and 24 or 25 percent were judged as poor 
or very poor. 

For operational projects 

In final appraisal of the 43 projects, five projects were rated 
as outstanding, and 20 were judged as good. Thus, a total of 25 pro- 
jects were considered outstanding or good. (Since all of the above 
grants were terminated, the evaluators did not include such items as 
failure to plan for continuation or provisions for adequate evaluations 
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as increments in their evaluation. ) The major concerns were identify- 
ing program objectives and how well these were accomplished. Of 
course, the apparent contributions and changes in the schools and the 
students as a result of the Title III project were also important consid- 
erations. 

The following table summarizes the final appraisal of the 137 



projects: 



Rating 


Planning 


Operational 


Total 


a. outstanding 


14 


5 


19 


b, good 


37 


20 


57 


c, average 


19 


11 


30 


d, poor 


18 


7 


25 


e, very poor 


6 


0 


6 


Totals 


94 


43 


137 



nJ ^ 



m 
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as increments in their evaluation. ) The major concerns were identify- 
ing program objectives and how well these were accomplished. Of 
course, the apparent contributions and changes in the schools and the 
students as a result of the Title III project were also important consid- 
erations. 



The following table summarizes the final appraisal of the 137 



projects: 








Rating 


Planning 


Operational 


Total 


a. outstanding 


14 


5 


19 


b. good 


37 


20 


57 


c. average 


19 


11 


30 


d. poor 


18 


7 


25 


e. very poor 


6 


__0 


6 


Totals 


94 


43 


137 



R ECOMMEND ATIONS 



Based upon this study, the following recommendations for fu- 
ture action are suggested: 

I, EVERY PROJECT PROPOSAL SHOULD BE REQUIRED TO 
SUBMIT EVIDENCE THAT THOSE DEVELOPING THE PRO- 
JECT HAVE A GOOD GRASP OF THE LOCAL AREA , BOTH 
IN TERMS OF NEEDS AND RESOURCES . 

This evidence does not have to be a needs assessment 
study if the project is a single idea or a program; still, some evi- 
dence of local considerations should be evident. 

II. EVERY PROPOSED SUPPLEMENTARY CENTER TYPE OF 

pace program should include a reasonably 

THOROUGH NEEDS ASSESSMENT STUDY . 

A statement or assessment of needs should include the 
identification of goals, processes for goal attainment, and specifica- 
tion of areas of greatest needs and deficiencies. Most terminal re- 
ports included little or nothing about how issues or programs were 
selected, and, in many other instances, this essential aspect of the re- 
port covering the supplementary center type of project, was either 
minimized or omitted. 
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[II. STATE DEPARTMENTS OF EDUCATION NEED TO GIVE 

|3AREFUL CONSIDERATION TO THE TYPE OF TERMINAL 
REPORTS THAT WILL PROVIDE A FITTING CLIMAX TO A 
PACE PROJECT . WILL MEET LEGAL REQUIREMENTS OF 
REPORTING . AND WILL ALLOW ESSENTIAL FINDINGS TQ_ 

BE DISSEMINATED EFFECTIVELY . 

^ Our study found that most projects omitted one or more 
types of information, such as: project title, type of project, grant 

number, period of time, amount of the grant, number of students to be 
served, cost per student, number of school~4i.stricts involved, the 

name of the state, and so forth. 

The study team had no idea how sloppy, inaccurate, and in- 
complete it would find the final reports, in most cases . Those who 
submit such reports are guilty of professional negligence and fiscal ir- 
responsibility, and they need to be dealt with accordingly. If this 
message seems overstated, one needs only to examine the end of pro- 
ject reports submitted by many projects. 

IV. INVOLVEMENT OF COMMUNITY RESOURCES AND PERSON_-_ 
NEL SHOULD BE MORE CAREFULLY CONSIDERED ; 

SHOULD BE REALISTIC AND SHOULD HAVE ADEQUATE 
FOLLOW - THROUGH . 

No promises should be made that cannot be kept; no obli- 
gations should be incurred that cannot be met. 
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A majority of the projects studied are guilty of overexten- 
sion and superficiality on community relations. The project developers 
promise too much, involve too many, and analyze too little the HOW of 
effective community involvement. 

V. ALL PROJECTS SHOULD HAVE EFFECTIVE EVALUATION 
PROCEDURES -— EFFECTIVE IN TERMS OF STATED OBJEC- 
TIVES AND PLANNED PROGRAMS. 

The call for better evaluation is an old saw, if the three 
years of PACE history is old, but the call needs to be made again and 
again. 

In only one or two instances out of 94 planning projects can 
one glean from the proposal a serious and sophisticated concern about 
evaluation — a concern that viewed evaluation as a vital part of the day- 
to-day monitoring process as well as a judgmental decision reflecting 
success or failure of the program. 

VI. EVERY PACE PROPOSAL SHOULD HAVE A SEPARATE BUD- 
GET ITEM FOR EVALUATION , AND THIS FIGURE SHOULD 
NOT BE LESS THAN FIVE PERCENT OF THE TOTAL BUD- 
GET . 

Only a small number of the terminated projects included 
plans for evaluation, and even these appeared to be afterthoughts or 
were non-integral parts of the project structure. While there is rea- 
son to believe that evaluation has improved during the last year, the 
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level is still far below what is desirable and whatsis needed. 

VII. MORE EVIDENCE OF PLANNING SHOULD BE REQUIRED IN 
FUTURE PACE PROPOSALS . 

The study team was aware of the dilemma between over- 

f 

planning and over- structuring on the one hand, and a relaxed, prag- 
matic approach to design on the othfer, but evidence gained from the 
137 terminal reports lead as to believe that greater emphasis should 

be placed on planning and design in all future proposals. 

/ 

VIII. PROVISIONS FOR CONTINUATION AFTER TERMINATION OF 

ESEA title m funding should become more EVIDEN T 
m THE FUTURE . 

The newness of PACE, the unexplored parameters of its 
guidelines, and the unknown labyrinths of federal assistance have all 
mitigated against serious consideration of what might take place when 
the planning grant ended. But as we look ahead, profiting from the 
past, continuation considerations should become more important with- 
out becoming a requirement for approval. 

IX. FUTURE PACE PLANNING GRANTS SHOULD BE ALLOCATED 
ON A SHARING BASIS WITH LOCAL COMMUNITIES -— SOME- 
THING IN THE DOLLAR RANGE OF 8 OR 10 ONE . 

The study team foumd that where local funds were com- 
mitted to the project, it was better planned, the objectives more 
clearly stated, and the procedures for reali.z,ing the major goals of the 
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project more adequately outlined. Furthermore, the study team be- 
lieved that a local share of the project expenses — even if small — en- 
hanced prospects of local continuation after termination of federal 
money. Also, dissemination and implementation are expedited by a 
sharing basis, and a more receptive climate is created for continua 
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EVALUATION INSTRUMENT 
Analysis of 

Terminated ESEA Title III Operational Programs 

A. Project Char '^.cteri sties 

1. Grant number 

2. Period of projec t 

3. Total amount of grant 

4. Total Non-Federal support 

5. Total Federal support under Title III 

6. Total Federal support other than Title III 

7. Major description of project (check one only) 
a. Innovative 

b. Exemplary 

c. Adaptive 

8. Types of activity (check one or more) 

a. Planning of program 

b. Planning of construction 

c. Conducting pilot activities 
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Operation of program. 

Constructing 

Remodeling 
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d. 



e. 



f. 



9. Project title (5 words or less) 



10. State of applicant 

11. Scope of project (check one only) 

a. National 

b. Regional (multi- state) 

c. One state 

d. One district 

e. Multi-district; meaning two to six districts 

f. Multi-district (or county); meaning seven or 

more districts within one state 

g. A few schools within one district 

h. One school 

I 

12. Area served by project (best fit one) 

a. Urban 

(1) central city 

(2) urban fringe 

b. Other urban (suburban) 

(1) cities of 10, 000 or more 

(2) cities of 2, 500 to 10, 000 



•■'>‘1 V 



- 58 - 



c. Rural 



(1) places of 1, 000 to 2, 500 

V 

# 

(2) other rural 

13. Type of activity (check only one) 

a. Mostly services to several local school districts, 
such as administration, dissemination, planning, 
and developing. 

b. Mostly instruction or services to pupils, such as 
arts, sciences, social studies, mobile demon- 
strations, museum visits, ETV , outdoor camp- 
ing, and guidance and counseling. 

c. Mostly services to teachers and principals 
services such as instructional materials, 
audio-visual, demonstrations, and in-service 
training. 

d. Mostly planning, such as surveying needs, de- 
signing new programs, and visiting innovations. 

e. Mostly installation of one or two innovations in 
one or two schools — innovations such as ITA, 
computer-assisted instruction, teaching aides, 
jparent involvement, job placement, and new 
courses. 

14. Project classification 

a. Focusing upon a single idea or program. 

b. Serving as a supplementary education center 

with several activities. 



15. School enrollment, etc. (Section C in guidelines) 
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16. Number of persons served and estimated cost distribution 
(Section E in guidelines) 



B. Project Accomplishments (What do the authors contend was 

achieved? What appraisal of this 
is made by reviewer? ) 



1. Research 



Accomplishment Appraisal 

1 . 

% 

2 . 

3. 

4. . ; 

2. Product development (such as new program for social 

studies, new materials for hahdi- 
capped) 

Accomplishment Appraisal 

1 . 

2 . ' 



3 . 



ilipililpip 
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3. Skill development (such as reading, mathematics, voca- 
tional) 

Accomplishment Apprais^ 

1 . 

2 . 

3. 




4. 

4, Process development (such as inquiry training) 

Accomplishment Appraisal 

1 . 

2 . 

3. 

4. 

5. Program development (such as nongraded school, team 

teaching) 

Accomplishment Appraisal 

1 . 

2 . 

3 . 

4. 










m 
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6. Pupil relations (such as guidance and counseling) 

Accomplishment Appraisal 

1 . 

2 . 

3. 

4. 

7. Community relations (as fostered by project) 

Accomplishment Appraisal 

1 . 

2 . 

3. 

4. 

8 . In- service programs for teacher s and / or admini s t r ato r s 

Accomplishment Appraisal 

1 . 

2 . 

3. 

4. 

9. Demonstration (of a program, such as team teaching) 

Accomplishment Appraisal 



1 
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10. Dissemination (spreading "the word" to others) 

Accomplishment Appraisal 

1 . 

2 . 

3. 

4. 

1 1 . Impl erne ntati o n 

Accomplishment Appraisal 

1 . 

2 . 

3. 



4. 



C . Provisions for Continuation 

1. (yes or no) Does the project give at least some 

consideration to continuation by means 
other than support by ESEA Title III. 

2. If yes, are these considerations- - 

a. well planned and likely to 

succeed 

b. questionable 




! 














c. quite sketchy 

d. too nebulous to appraise 
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m 
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Design of Project as Evidenced from Reporj 

1. Degree to which evaluation procedures are related 
to the objectives 

a. (much), b. (some), c. (little), d. (no evi- 
dence of relationship) 

2. What evaluative procedures are used? (research 
design; pre-test, post-test, survey, interview, 
etc. ) 



3. Estimated cost of evaluation 

4. What relationship does evaluation appear to have in 
terms of the overall project? 



a. 


An integral part 


b. 


A part 


c. 


Not any particular relationship, or very 
casual 


d. 


Evaluation appears forced to need the 
guidelines requirement 



Final Appraisal 

1. What one or two ideas, programs, etc., do you consider 
significant and worthy of widespread dissemination? 



2. From your study of the report, what were two or three 
major problems that seemed to be implicit or explicit? 
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What is your overall appraisal of the project as gleaned 
from the report? 



a. 


outstanding 


b. 


good 


c. 


average 


d. 


poor 


e. 


very poor 



1 







































Addendum for Terminated Planning Grants 



Planning projects will focus on (1) an extensive needs assess- 
ment, or (2) upon the development of a particular idea or program. 

1 . Needs assessment 

a. (yes or no) Did the planning grant include a needs 

assessment study? 

b. If answer is "yes, " then respond to the following 
items: 

1. Which of these data were used in the assess- 
ment? (check appropriate ones) 

(a) Community factors 



(i) 


Rural-urban composition 


(n) 


Median family income 


(iii) 


Median grade attained 


(iv) 


Socio-economic class 




structure 


(v) 


Employment patterns 


(vi) 


Horizontal mobility 


(vii) 


Per pupil expenditure 


(viii) 


Attitudinal considerations 


(ix) 


Other 



(b) School factors 

(i) Student achievement mea- 
sures 

(ii) Socio-economic class 

structure of school sys- 
tem 

(iii) Composition of student 

body 

(iv) Other 
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2. What procedures were employed to determine 
needs? (check appropriate ones and write in 
other s) 

(a) Questionnaire 

(b) Interview 

(c) Discussion groups 

(d) Study groups 

(e) Research design 

( f) Other 



Who determined the needs? 


(a) 


Members of present staff 


(b) 


New personnel 


(«) 


Special study team 


(d) 


"Outside" consultants 


(e) 


Other 



4. Is systematic planning in evidence? 

(a) Yes 

(b) No 

(c) Yes and no 

(d) Cannot tell 

5. Are any planning procedures used, such as: 

(a) system analysis 

(b) PERT ' 

(c) Critical Path Method 

(d) sequences or phases 

6. Are evaluation procedures built into 1;he plan- 
ning procedures? yes no 

Development of ^ particular idea or program 



Can you perceive what the project has in mind? 


(1) 


yes, clearly 


(2) 


yes 


(3) 


somewhat 


(4) 


little clarity 


(5) 


for all practical purposes, I cannot 
perceive this from the report. 












-69- 



b. Has the report presented the idea or program in a 
concise and understandable manner? 

(1) yes, clearly 

(2) yes 

(3) somewhat 

(4) little clarity 

(5) for all practical purposes, the presen- 
tation shall be classified as imprecise 
and confusing. 



The purpose of a planning grant is to develop a proposal that 



can be turned into an operational grant. In about 75 percent of the 



cases this has happened. 

3 . Based upon this report , what kind of ^ picture do you have 

about the next stage — the operational grant ? 



(a) clear picture 

(b) some clarity 

(c) uncertain 

(d) very little clarity about operational 
phase 

(e) evidently no operational grant is 
planned 



